Camera Calibration

ABSTRACT

Online camera calibration methods have been proposed whereby calibration information is extracted from the images that the system captures during normal operation and is used to continually update system parameters. However, such existing methods do not cope well with structure-poor scenes having little texture and/or 3D structure such as in a home or office environment. By considering camera families (a set of cameras that are manufactured at least partially in a common manner) it is possible to provide calibration methods which are suitable for use with structure-poor scenes. A prior distribution of camera parameters for a family of cameras is estimated and used to obtain accurate calibration results for individual cameras of the camera family even where the calibration is carried out online, in an environment which is structure-poor.

BACKGROUND

Cameras are used to capture images of scenes and in order to make use ofthe captured images, for example, to navigate a robotic vehicle, to pickup items in a robotic manufacturing system or to estimate distances ofobjects from a stereo web camera, it is often necessary to havecalibration information for the camera being used. For example, thiscamera calibration information, for a mono-camera, may be a set of fiveparameters, one for the focal length of the camera, one for the pixelaspect ratio, two for the image coordinates of the principal point, andone parameter describing nonlinear lens distortion.

It is known to calibrate a camera in a laboratory environment bypresenting a known target or motion sequence to the camera. However,this is time consuming and expensive. Also, re-calibration is typicallynecessary because even camera systems which have been constructed totight tolerances from high-quality components eventually change theircharacteristics. For example, this might be due to changes intemperature, dropping or jolting the camera, or changes in humidity. Inorder to re-calibrate the camera an explicit calibration procedure in alaboratory environment can be repeated but this is expensive and timeconsuming as mentioned above. Also, this type of calibration process isnot suitable for systems such as autonomous robots, which must work inunconstrained environments, or for mass-market consumer applications.

Online camera calibration methods have been proposed whereby calibrationinformation is extracted from the images that the system captures duringnormal operation and is used to continually update system parameters.However, existing online camera calibration methods do not cope wellwith structure-poor scenes; that is, where the images that the systemcaptures during normal operation comprise little texture and/or 3Dstructure. Where cameras are used in the home or office environment itis often the case that images captured in normal operation arestructure-poor and/or comprise features which hinder online calibration.

SUMMARY

The following presents a simplified summary of the disclosure in orderto provide a basic understanding to the reader. This summary is not anextensive overview of the disclosure and it does not identifykey/critical elements of the invention or delineate the scope of theinvention. Its sole purpose is to present some concepts disclosed hereinin a simplified form as a prelude to the more detailed description thatis presented later.

Online camera calibration methods have been proposed whereby calibrationinformation is extracted from the images that the system captures duringnormal operation and is used to continually update system parameters.However, such existing methods do not cope well with structure-poorscenes having little texture and/or 3D structure such as in a home oroffice environment. By considering camera families (a set of camerasthat are manufactured at least partially in a common manner) it ispossible to provide calibration methods which are suitable for use withstructure-poor scenes. A prior distribution of camera parameters for afamily of cameras is estimated and used to obtain accurate calibrationresults for individual cameras of the camera family even where thecalibration is carried out online, in an environment which isstructure-poor

Many of the attendant features will be more readily appreciated as thesame becomes better understood by reference to the following detaileddescription considered in connection with the accompanying drawings.

DESCRIPTION OF THE DRAWINGS

The present description will be better understood from the followingdetailed description read in light of the accompanying drawings,wherein:

FIG. 1 is a block diagram of an example method of estimating a priordistribution of camera parameters of a family of cameras;

FIG. 2 is a block diagram of an example method of estimating cameraparameters of a camera system using a stored prior distribution;

FIG. 3 is a block diagram of another example method of estimating aprior distribution of camera parameters of a family of cameras;

FIG. 4 is a schematic diagram of a stereo camera to be calibrated in ahome environment;

FIG. 5 illustrates an exemplary computing-based device in whichembodiments of a camera calibration system may be implemented.

Like reference numerals are used to designate like parts in theaccompanying drawings.

DETAILED DESCRIPTION

The detailed description provided below in connection with the appendeddrawings is intended as a description of the present examples and is notintended to represent the only forms in which the present example may beconstructed or utilized. The description sets forth the functions of theexample and the sequence of steps for constructing and operating theexample. However, the same or equivalent functions and sequences may beaccomplished by different examples.

As mentioned above, existing online calibration methods are best suitedto scenes with rich 3D structure and with plentiful texture for matching(in the case of stereo camera systems). However, such scenes are oftenunavailable, especially in the home or office environment. In thesesituations there are often a number of challenges to online calibration,such as repeated structure, specular reflections, large areas of notexture, and often a lack of depth relief. In many cases, imagescaptured for online calibration not only lack rich structure, but alsoinclude clusters of outlier data which conspire to increase thelikelihood of incorrect solutions (e.g. wallpaper with a repeatingpattern or Venetian blinds).

By considering camera families it is possible to provide onlinecalibration methods which are suitable for use with structure-poorscenes. The term “camera family” is used to refer to a set of cameraswhich are manufactured at least partially in a common manner. Forexample, a camera family may be all cameras of the same model made by aparticular manufacturer at a particular manufacturing plant. Anotherexample of a camera family is the set of cameras made by the samemanufacturer. Another example might be the set of cameras made using thesame manufacturing technique by different manufacturers. In high levelterms, the methods described herein comprise estimating a priordistribution of camera parameters for a family of cameras. This priordistribution may then be used to obtain accurate calibration results forindividual cameras of the camera family even where that calibration iscarried out online, in an environment which is structure-poor.

FIG. 1 is block diagram of an example method of estimating a priordistribution of camera parameters for a family of cameras. This is adifficult task because it is not practical to obtain hundreds of camerasfrom the family and calibrate each of those using an offline,laboratory-based calibration method. In addition, the parameters that itis required to calibrate exist in a relatively high-dimensional space.For example, for stereo cameras a minimal parameterization has sevendegrees of freedom. Because of this a large number of sample cameraswould need to be calibrated in order to obtain a reliable estimate ofthe prior distribution for the camera family. However, it is veryimportant to obtain a good estimate or model of the prior distributionfor the camera family in order to enable accurate calibration ofindividual members of the camera family. The methods described hereinenable a good estimate of the prior distribution for the camera familyto be obtained without the need for calibrating hundreds of cameras fromthe family in an offline laboratory-based process.

As shown in block 100 of FIG. 1 a plurality of sample cameras areobtained from a camera family. For example, seven sample cameras may beused in the case of a camera family of stereo cameras formed from pairsof mono web-cameras fixed together. Alternatively a few dozen samplecameras may be used. The number of sample cameras required depends onthe characteristics of the camera family, such as the number ofdimensions of the parameters of the cameras in the family. Dependingfurther on the method used to represent the prior distribution,different numbers of sample cameras can be used, from a minimum of two,with no maximum. The sample cameras are then calibrated (block 101)using an offline process which may require laboratory environment. Anysuitable offline camera calibration process may be used to obtainestimates of the camera parameters for each sample camera. Examples ofsuitable offline camera calibration processes are described in detail inHartley and Zisserman “Multiple View Geometry in Computer Vision” secondedition, 2003, Cambridge University Press, ISBM 0521 54051 8.

The calibration results for the sample cameras are then used in aprobability density estimation process to estimate a prior distributionof camera parameters for the camera family (block 102). Any suitableprobability density estimation process may be used. For example, akernel density estimator is constructed using the sample camera resultsas described in more detail below. Alternative probability densityestimation processes include the fitting of a parametrized model such asa Gaussian or a mixture of Gaussians, in which the forms of theGaussians may optionally be restricted to have specified special formsof covariance matrices, for example diagonal, scaled identity, blockdiagonal, etc. The estimated prior distribution is then stored (block103). In one example, the probability density estimation processcomprises fitting a Gaussian with full covariance matrix to the samplecamera parameters under a Wishart prior on the covariance. FIG. 2 is ablock diagram of an example method of estimating camera parameters of acamera system. A prior distribution for the camera family has alreadybeen estimated in an offline process as described above and is stored inan accessible database or other memory (block 200). A camera to becalibrated in an online process, which may be a stereo camera or othercamera system depending on the camera family concerned, is set up in astructure-poor environment such as a home or office environment (block201). The camera is then used to capture one or more images of astructure-poor scene (block 202). For example, if the camera is a stereocamera at least one pair of images forming a stereo pair are captured.In the case of a mono-camera at least two images are captured of aconstant scene with the camera in different positions for the twoimages. A novice user is able to operate the camera in order to capturethe images or this may be carried out automatically in a processtransparent to the end user.

Image points in the captured images are then identified (block 203)using any suitable process such as a corner detection algorithm, a blobdetection algorithm, a line detection algorithm or similar. For example,a Harris edge detector is used as described in “A combined corner andedge detector” Harris and Stephens, Proceedings of The Fourth AlveyVision Conference, Manchester, pp 147-151. 1988. Taking an image pair,image points that correspond in the two images of the pair areidentified (block 203) using any suitable matching process. For example,automatic homography matching as described in Hartley and Zisserman 2003referenced above, or SIFT matching as described in Lowe “Distinctiveimage features from scale-invariant keypoints”, International Journal ofComputer Vision, 60, 2 (2004) pp. 91-110, or by manually indicatingcorresponding points. The resulting correspondences are then used toestimate the camera parameters (block 204) and so to estimate alikelihood distribution. This distribution over the camera parameters isrelatively broad because the images are captured at block (202) in astructure-poor environment in a process carried out by a novice user.Because the images are captured in a structure-poor environment, only afew or perhaps only one point correspondence is identified which giveslittle information for the calibration process.

The prior distribution is accessed (block 200) and combined (block 205)with the likelihood distribution to produce a posterior distribution(block 206) following Bayes Rule. The posterior distribution is themmaximized to find parameter values for the camera being calibrated(block 207) using any suitable maximization process. The parametervalues for the camera are stored (block 208).

In this way, information about the camera family is used to improvecalibration of a camera from that family even where calibration has beencarried out in a structure-poor environment such that perhaps only onepoint correspondence is obtained during the calibration process.

FIG. 3 is a block diagram of another example method of estimating aprior distribution of camera parameters for a family of cameras. Asmentioned above, in one example, a kernel density estimator isconstructed using the sample cameras. For example, a set of dense pointcorrespondences is accessed for each sample camera from the offlinecalibration process (block 300). A Gaussian approximation of a priordistribution for each sample camera is made using estimates of thecamera parameters obtained from the point correspondences (block 301).The individual camera prior distributions are then combined to obtain acamera-family prior (block 302). In this way a strong estimate of theprior distribution is obtained which may be constructed from only asmall number of sample camera results.

During the online camera calibration process it is also possible for theend user to hold a target display in the camera's field of view. Imagescaptured by the camera of this target display may then be used in theonline calibration process thus making the process of identifying imagepoints and matching those points more straightforward. For example, FIG.4 shows a personal computer (PC) 400 with a display monitor having astereo camera 401 placed on top of the PC monitor. A person 403 holds atarget display 402 in the field of view of the camera as shown.

Detailed examples of methods of computing the prior distribution for thecamera family and of calibrating a camera from that family are nowgiven. Notation that will be used is first explained.

The term ‘camera system’ is used herein to refer to one or more camerasarranged together to form an image capture apparatus. The cameras may beconventional cameras, bespoke cameras with special lens arrangements andmay comprise stereo rigs and the like.

There is a camera system characterized by parameters θ, which might be,for example, the nine entries of the fundamental matrix of a stereo rig,or perhaps seven parameters describing the focal length and 3D positionof a security camera. In the case of a stereo camera rig we choose touse the internal camera parameters for the two cameras comprising therig, and the rotation and translation between the two.

The available information is a set of N observations Z={z_(i)}_(i-1)^(N), where in this work z_(i) are point correspondences between twoimages, represented as 4-vectors z=(x,y,x′,y′). Associated with eachobservation is a latent variable x_(i), which, in the stereo cameracase, will be the 3D point which gave rise to the correspondence.Finally there is a projection function ƒ(θ,x) which generates noiselessobservations, to which are added noise ν drawn from some robustdistribution with density function p(ν) ∝ exp(−ρ(ν)), defined by arobust kernel p(•), e.g. the Lorentzian ρ(ν)=log (π(1+∥ν∥²). Thus

z=ƒ(θ,x)+ν  (1)

is the observation model.

The goal is to maintain an estimate of θ, and in particular thedistribution of θ conditioned on the observations, p(θ|Z). When theparameters are required it is possible to compute the mode of thisdistribution {circumflex over (θ)}=argmax₀p(θ|Z) so the distributionneed be maintained only up to an overall scale. (Note the symbol ̂ isused to indicate an estimator of a parameter. For example {circumflexover (θ)} is an estimate of the camera parameters.) By Bayes' rule,p(θ|Z) is expanded to the product of likelihood and prior:

p(θ|Z) ∝ p(Z|θ)p(θ),   (2)

The likelihood p(Z|θ), with the noise assumed independent, andoptimizing over the latent variables x is given as:

$\begin{matrix}{{{- \log}\; p\text{(}Z\left. \theta \right)} = {\sum\limits_{i = 1}^{N}{\min\limits_{x}{{\rho \left( {z_{i} - {f\left( {\theta,x} \right)}} \right)}.}}}} & (3)\end{matrix}$

Online calibration may proceed by capturing a (small) set of pointcorrespondences, Z_(online), and then computing the parameters whichmaximize the posterior

$\begin{matrix}{{\hat{\theta}}_{online} = {\underset{\theta}{\arg \; \max}{p\left( {Z_{online}\left. \theta \right){{p_{F}(\theta)}.}} \right.}}} & (4)\end{matrix}$

Note that the prior over θ is now written p_(F) to indicate that it is aprior over a camera family rather than over all cameras. A standardnonlinear optimizer may be used to perform this maximization, because agood initial estimate may be provided.

As mentioned above, the parameters to be calibrated live in a relativelyhigh-dimensional space. For stereo cameras, it is required to calibratethe fundamental matrix between the two image planes: a minimalparameterization has seven degrees of freedom, so θ ε R⁷. To fit asingle Gaussian with full covariance to the samples requires a minimumof eight sample cameras, but in practice many times this number would beneeded in order to obtain a reliable estimate.

Furthermore, the distribution of parameters in such minimalparameterizations are often strongly nonlinear and poorly fit by asingle Gaussian. This is mitigated by using a higher dimensionaloverparameterization as described below, in which space the Gaussianmodel is likely to prove a better fit.

In an example a kernel density estimator is formed using the samplecameras. Given M cameras, for each camera a large, space-filling set ofcorrespondences Z_(m)={z_(mi)}_(i=1) ^(N) ^(m) is obtained. A Gaussianapproximation of p(θ|Z_(m)) is made by finding the mode {circumflex over(θ)}_(m) and taking the second-order expansion of p(θ|Z_(m)) around themode:

$\begin{matrix}{{\hat{\theta}}_{m} = {\underset{\theta}{\arg \; \max}{p\left( {Z_{m}\left. \theta \right){p(\theta)}} \right.}}} & (5) \\{{p\text{(}\theta \left. Z_{m} \right)} \approx {N\left( {\theta \left. {{\hat{\theta}}_{m},{\hat{\Sigma}}_{m}} \right)} \right.}} & (6)\end{matrix}$

The global camera prior p(θ) is a broad Gaussian with parameters asdiscussed below, and {circumflex over (Σ)}_(m) is the covariance matrix,computed as below.

The camera-family prior p_(F)(θ) is then obtained by combining thesedistributions over θ. The prior may be approximated by a Gaussian,having chosen a parameterization (see below) which makes this a goodapproximation. The prior parameters will be written θ and Σ, giving

p _(F)(θ)=N(θ| θ, Σ)   (7)

Various approximations may be used for computation of θ and Σ. Anexample approximation is to assume that the datasets were independentdraws from the camera family so that

$\begin{matrix}{{p_{F}(\theta)} = {\prod\limits_{m = 1}^{M}\; {p\left( {\theta {\left. {Zm} \right)^{\eta}.}} \right.}}} & (8)\end{matrix}$

Where η is a tuning factor which can be empirically set.

As described above, two stages are specified: the first dealing withoffline calibration and the construction of p_(F); the second describingthe online process. In both stages, a common tool is the optimization ofthe posterior (2) under a Gaussian prior, which will be denotedgenerically by the negative log posterior function ε(θ;Z,μ,Σ), definedby

$\begin{matrix}{{ɛ\left( {{\theta;Z},\mu,\Sigma} \right)} = {{\sum\limits_{i = 1}^{N}{\min\limits_{x}{\rho \left( {z_{i} - {f\left( {\theta,x} \right)}} \right)}}} + {\left( {\theta - \mu} \right)^{T}{\Sigma^{- 1}\left( {\theta - \mu} \right)}}}} & (10)\end{matrix}$

where additive constants independent of θ have been omitted.

A particular example with reference to a family of stereo cameras is nowgiven.

Parameterization

Five intrinsic parameters may be used for each camera: focal lengthsalong x and y directions (f_(x) and f_(y)), principal point (p_(x) andp_(y)) and skew s. The motion between the two cameras in a stereo rig isgiven by 3 rotational and 2 translational parameters. The rotation isrepresented by a three-parameter Rodrigues vector ω, with which istypically associated a base rotation matrix R₀ as described in B.Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon “Bundleadjustment—a modern synthesis” Vision algorithms: Theory and practice,Springer-Verlag LNCS 1883, 2000. Let function rod (ω) denote the 3×3matrix representation of the rotation defined by ω. Translation ismodelled as two rotations about fixed axes a₁,a₂ perpendicular to a basetranslation t₀, giving t(α₁,α₂)=rod(α₁a₁)rod(α₂a₂)t₀. Three-dimensionalpoints x are parameterized by homogeneous 4-vectors, with thegauge-fixing constraint that ∥x∥=1. Thus the 15-element parameter vectoris

$\begin{matrix}{\theta = \left\lbrack {f_{x},\frac{f_{x}}{f_{y}},s,p_{x},p_{y},f_{x}^{\prime},\frac{f_{x}^{\prime}}{f_{y}^{\prime}},s^{\prime},p_{x}^{\prime},p_{y}^{\prime},\omega_{1},\omega_{2},\omega_{3},\alpha_{1},\alpha_{2}} \right\rbrack^{T}} & (11)\end{matrix}$

where primed quantities are in the second camera. The benefit of thisparameterization is that we can use a single Gaussian to model the priorover θ.

Given this parameterization, the observation function ƒ may be defined.Recall that ƒ takes parameters θ, a 3D point x and generates a singlecorrespondence (x,y,x′,y′), which is defined by

$\begin{matrix}{\begin{pmatrix}x \\y \\1\end{pmatrix} \sim {\begin{pmatrix}f_{x} & {sf}_{x} & p_{x} \\0 & f_{y} & p_{y} \\0 & 0 & 1\end{pmatrix}\text{[}I\left. 0 \right\rbrack {x\begin{pmatrix}x^{\prime} \\y^{\prime} \\1\end{pmatrix}}} \sim {\begin{pmatrix}f_{x}^{\prime} & {s^{\prime}f_{x}^{\prime}} & p_{x}^{\prime} \\0 & f_{y}^{\prime} & p_{y}^{\prime} \\0 & 0 & 1\end{pmatrix}\left\lbrack {{{rod}(\omega)}R_{0}\left. {t\left( {\alpha_{1},\alpha_{2}} \right)} \right\rbrack x} \right.}} & (12)\end{matrix}$

where ˜ denotes equality up to scale.

Offline Calibration

In the offline calibration of the m^(th) stereo camera, densecorrespondences are obtained by moving a large planar textured targetrelative to the camera. This typically gives rise to several hundredpoint correspondences, denoted Z_(m), from which initial estimates of θare obtained using RANSAC and self-calibration as described in R.Hartley and A. Zisserman “Multiple View Geometry in Computer Vision”,Cambridge University Press, 2000. These are refined by nonlinearoptimization of (10), viz:

θ_(m)=argmin ε(θ;Z_(m),μ,Σ)   (13)

under a broad global prior, where, for example, μ=(800, 1, 0, 320, 240,800, 1, 0, 320, 240, 0, 0, 0, 0, 0); Σ=diag(300, 0.005, 10⁻⁴, 25, 25,300, 0.005, 10⁻⁴, 25, 25, ∞, ∞, ∞, 1, 1)². These are expressed in pixelcoordinates, but all calculations are scaled so as to work inconditioned coordinates. The Gaussian approximation of the m^(th) cameraposterior p(θ|Z_(m)) is then given [C. Bishop “Pattern recognition andmachine learning”, Springer, 2006, p 214ff] by μ_(m)=θ_(m), andΣ_(m)=H⁻¹ where H is the Hessian of ε, with ij^(th) entry

$\begin{matrix}{H_{ij} = {\frac{\partial}{\partial\theta_{j}}\frac{\partial ɛ}{\partial\theta_{j}}}} & (14)\end{matrix}$

Because the Hessian is not necessarily positive definite after numericalminimization, all negative eigenvalues are replaced with zero; and wheninverting, the pseudoinverse is used.

Computing the Prior

Combining the per-camera posteriors using (8) then gives the followingexpression for a Gaussian prior over the camera family.

$\begin{matrix}{\overset{\_}{\theta} = {\frac{1}{M}{\sum\limits_{m = 1}^{M}\theta_{m}}}} & (15) \\{\overset{\_}{\Sigma} = \left( {\eta {\sum\limits_{m = 1}^{M}\Sigma_{m}^{- 1}}} \right)} & (16)\end{matrix}$

For example, η may be set to 0.1 or any other available value.

It is also possible to truncate Σ to low rank before use in the prior.

Online Calibration

Given the parameters of the prior, online estimation is a matter ofoptimizing the error function (10) with the prior just described. In theextreme case of a single point correspondence Z={z₁} corrupted byGaussian noise, the estimate is given by

$\begin{matrix}{\overset{\_}{\theta} = {\underset{\theta}{\arg \; \min}\mspace{11mu} {ɛ\left( {{\theta;\left\{ z_{1} \right\}},\overset{\_}{\theta},\overset{\_}{\Sigma}} \right)}}} & (17) \\{\mspace{11mu} {= {\underset{\theta}{\arg \; \min}\; \left\lfloor {{\min\limits_{x}{{z_{1} - {f\left( {\theta,x} \right)}}}^{2}} + {\left( {\theta - \overset{\_}{\theta}} \right)^{T}{{\overset{\_}{\Sigma}}^{- 1}\left( {\theta - \overset{\_}{\theta}} \right)}}} \right\rfloor}}} & (18)\end{matrix}$

This may be implemented using MATLAB's optimization routine fminunc onthe equivalent problem

$\begin{matrix}{{{\min\limits_{\theta,x}{{z_{1} - {f\left( {\theta,x} \right)}}}^{2}} + {\left( {\theta - \overset{\_}{\theta}} \right)^{T}{{\overset{\_}{\Sigma}}^{- 1}\left( {\theta - \overset{\_}{\theta}} \right)}}},} & (19)\end{matrix}$

with θ as initial estimate, and initial estimate of x obtained usingoptimal triangulation of the correspondences. Optimal triangulation isdescribed in Hartley and Zisserman referenced above.

Exemplary Computing-Based Device

FIG. 5 illustrates various components of an exemplary computing-baseddevice 500 which may be implemented as any form of a computing and/orelectronic device, and in which embodiments of a camera calibrationsystem may be implemented.

The computing-based device 500 may be integral with a camera or may beindependent and arranged to receive images captured by a camera. Itcomprises an input 501 which is of any suitable type for receivingimages and optionally other inputs 507 for receiving input of anysuitable type including information about a prior distribution for acamera family.

Computing-based device 500 also comprises one or more processors 503which may be microprocessors, controllers or any other suitable type ofprocessors for processing computing executable instructions to controlthe operation of the device in order to calibrate a camera of a camerafamily. Platform software comprising an operating system 504 or anyother suitable platform software may be provided at the computing-baseddevice to enable application software 505 to be executed on the device.

The computer executable instructions may be provided using anycomputer-readable media, such as memory 506. The memory is of anysuitable type such as random access memory (RAM), a disk storage deviceof any type such as a magnetic or optical storage device, a hard diskdrive, or a CD, DVD or other disc drive. Flash memory, EPROM or EEPROMmay also be used.

An output 508 is also provided such as an audio and/or video output to adisplay system integral with or in communication with thecomputing-based device. The display system may provide a graphical userinterface, or other user interface of any suitable type although this isnot essential. The output 508 may also provide camera parametersobtained by the camera calibration process to an image processing systemor other system arranged to use images captured by the camera.

The term ‘computer’ is used herein to refer to any device withprocessing capability such that it can execute instructions. Thoseskilled in the art will realize that such processing capabilities areincorporated into many different devices and therefore the term‘computer’ includes PCs, servers, mobile telephones, personal digitalassistants and many other devices.

The methods described herein may be performed by software in machinereadable form on a storage medium. The software can be suitable forexecution on a parallel processor or a serial processor such that themethod steps may be carried out in any suitable order, orsimultaneously.

This acknowledges that software can be a valuable, separately tradablecommodity. It is intended to encompass software, which runs on orcontrols “dumb” or standard hardware, to carry out the desiredfunctions. It is also intended to encompass software which “describes”or defines the configuration of hardware, such as HDL (hardwaredescription language) software, as is used for designing silicon chips,or for configuring universal programmable chips, to carry out desiredfunctions.

Those skilled in the art will realize that storage devices utilized tostore program instructions can be distributed across a network. Forexample, a remote computer may store an example of the process describedas software. A local or terminal computer may access the remote computerand download a part or all of the software to run the program.Alternatively, the local computer may download pieces of the software asneeded, or execute some software instructions at the local terminal andsome at the remote computer (or computer network). Those skilled in theart will also realize that by utilizing conventional techniques known tothose skilled in the art that all, or a portion of the softwareinstructions may be carried out by a dedicated circuit, such as a DSP,programmable logic array, or the like.

Any range or device value given herein may be extended or alteredwithout losing the effect sought, as will be apparent to the skilledperson.

It will be understood that the benefits and advantages described abovemay relate to one embodiment or may relate to several embodiments. Itwill further be understood that reference to ‘an’ item refer to one ormore of those items.

The steps of the methods described herein may be carried out in anysuitable order, or simultaneously where appropriate. Additionally,individual blocks may be deleted from any of the methods withoutdeparting from the spirit and scope of the subject matter describedherein.

It will be understood that the above description of a preferredembodiment is given by way of example only and that variousmodifications may be made by those skilled in the art. The abovespecification, examples and data provide a complete description of thestructure and use of exemplary embodiments of the invention. Althoughvarious embodiments of the invention have been described above with acertain degree of particularity, or with reference to one or moreindividual embodiments, those skilled in the art could make numerousalterations to the disclosed embodiments without departing from thespirit or scope of this invention.

1. A method comprising: obtaining a set of camera parameters for each ofa plurality of sample cameras, those sample cameras being members of afamily of cameras where all cameras of the family are manufactured atleast partially in a common manner; carrying out a probability densityestimation process to estimate a prior distribution of camera parametersfor the camera family using the camera parameters of the sample cameras;and storing the estimated prior distribution.
 2. A method as claimed inclaim 1 wherein the probability density estimation process comprisesforming a kernel density estimator.
 3. A method as claimed in claim 1wherein the probability density estimation process comprises fitting aGaussian with full covariance matrix to the camera parameters of thesample cameras under a Wishart prior on the covariance.
 4. A method asclaimed in claim 1 wherein the probability density estimation processcomprises estimating a prior distribution for each sample camera andcombining those to obtain the prior distribution of camera parametersfor the camera family.
 5. A method comprising: receiving at least twoimages of a scene captured at a camera to be calibrated, that camerabeing a member of a specified family of cameras where all members of thefamily of cameras are manufactured at least partly in a common manner;forming an initial estimate of parameters of the camera; accessing aprior distribution of camera parameters for the specified camera family,that prior distribution having been estimated by obtaining cameraparameters for a plurality of sample cameras from the camera family andcarrying out a probability density estimation process using the cameraparameters of the sample cameras; combining the initial estimate of theparameters of the camera with information from the received images, andthe accessed prior distribution to obtain a posterior distribution;maximizing the posterior distribution to find improved parameter valuesfor the camera; and storing the improved parameter values.
 6. A methodas claimed in claim 5 which comprises extracting information from thereceived images by identifying image points in the images and finding atleast one correspondence between an image point in one of the images andan image point in another of the images.
 7. A method as claimed in claim5 which comprises extracting information from the received images byidentifying image points in the images and finding only onecorrespondence between an image point in one of the images and an imagepoint in another of the images.
 8. A method as claimed in claim 5wherein the probability density estimation process comprises forming akernel density estimator.
 9. A method as claimed in claim 5 wherein theprobability density estimation process comprises fitting a Gaussian withfull covariance matrix to the camera parameters of the sample camerasunder a Wishart prior on the covariance.
 10. A method as claimed inclaim 5 wherein the probability density estimation process comprisesestimating a prior distribution for each sample camera and combiningthose to obtain the prior distribution of camera parameters for thecamera family.
 11. A method as claimed in claim 5 wherein the camera isa stereo camera comprising two cameras and wherein 15 camera parametersare used.
 12. A method as claimed in claim 5 wherein the images are of astructure-poor scene.
 13. A method as claimed in claim 5 wherein thestep of combining the initial estimate of the camera parameters withinformation about the accessed prior distribution comprises usingBayesian techniques.
 14. A camera calibration system comprising: aninput arranged to receive at least two images of a scene captured at acamera to be calibrated, that camera being a member of a specifiedfamily of cameras where all members of the family of cameras aremanufactured at least partly in a common manner; a processor arranged toform an initial estimate of parameters of the camera; a memory arrangedto store a prior distribution of camera parameters for the specifiedcamera family, that prior distribution having been estimated byobtaining camera parameters for a plurality of sample cameras from thecamera family and carrying out a probability density estimation processusing the camera parameters of the sample cameras; the processor alsobeing arranged to combine the initial estimate of the parameters of thecamera with information from the received images and the stored priordistribution to obtain a posterior distribution; the processor alsobeing arranged to maximize the posterior distribution to find improvedparameter values for the camera; and a memory arranged to store theimproved parameter values.
 15. A camera calibration system as claimed inclaim 14 wherein the processor is arranged to extract information fromthe received images by identifying image points in the images andfinding at least one correspondence between an image point in one of theimages and an image point in another of the images.
 16. A cameracalibration system as claimed in claim 14 wherein the processor isarranged to extract information from the received images by finding onlyone correspondence between an image point in one of the images and animage point in another of the images.
 17. A camera calibration system asclaimed in claim 14 wherein the processor is further arranged toestimate the prior distribution.
 18. A camera calibration system asclaimed in claim 14 for calibrating a stereo camera comprising twocameras and wherein the processor is arranged to use 15 cameraparameters.
 19. A camera calibration system as claimed in claim 14wherein the input is arranged to receive images of a structure-poorscene.
 20. A camera calibration system as claimed in claim 14 whereinthe processor is arranged to combine the initial estimate of the cameraparameters with information about the prior distribution using Bayesiantechniques.